Method and apparatus for user recognition using CCD cameras

ABSTRACT

A user recognition system that utilizes two CCD cameras to obtain two images of the user from two different angles of view. A three-dimensional model of the user&#39;s face is created from the obtained images in addition. The generated model and an additional facial texture image of the user are compared with a stored user profile. When the obtained 3D model and facial texture information matches the stored profile of the user, access is granted to the system.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to recognition systems, and moreparticularly to a method and apparatus for user recognition to grantaccess to computer systems using charged coupled device (CCD) cameras.

2. Description of the Related Art

Computer systems often have password-based recognition systems foridentifying the user and their authorization to access the system. Thepasswords are generally user selected and provide a “what you know” typestrategy to access the system, i.e., if you know the password you canaccess the system. These password systems are relatively unreliablebecause the user may disclose it to anyone else, either intentionally orunintentionally, so they may access the particular computer systemand/or service provided by the same. In addition, a user may write downhis password so as to not forget it, thereby providing the opportunityfor anyone to find and use it. A password is easily learned by actualdisclosure or by being viewed by another while the user is typing it into access the system. Thus, it is clear that a written password is alarge security risk as it may provide several people with access to asystem that only one person has been authorized to use.

A more secure recognition system focuses on who the user is, i.e.,characteristics of the user, as opposed to what the user knows. It wouldtherefore be preferable to provide a recognition system that is notbased on what the user knows, but rather is based on who the user is.

One example of a method of determining who the user is, is fingerprintidentification systems. These systems require the user to place theirfinger or hand on a dedicated scanner which will scan the finger orhandprint of the individual and compare the scanned print with thosestored in memory. Upon approval of the scanned print, the user will beprovided access to the computer system. It is understood that that thelevel of access provided may be limited or tailored to the specificindividual.

Other known systems utilize a single camera that takes a two-dimensionalimage of the user's face and compares it with stored 2-D images inmemory. For example, a system based on Connectix QuickCam is not capableof seeing a 3-dimensional object and therefore are limited to2-dimensional applications. Thus, it is apparent that the primarydrawback of such security system is that it must be capable of handlinga 2-D photograph of the authorized user.

Thus, it would be desirable to provide a user recognition system that iscapable of using 3-dimensional models of the user, and would thereforprovide a more secure system.

SUMMARY OF THE INVENTION

It is therefore an object of the invention to provide a method by whichthe user's identity is recognized by the system being accessed withoutrequiring the user to input a password.

It is yet another object of the invention to provide a method forrecognizing a user's identity for enabling access to a computer systemthat is based on a who the user is rather than what the user knows(i.e., an input password).

These and other objects are achieved in accordance with the method forrecognizing a user of the present invention wherein at least two2-dimensional still images of the user are obtained from at least twodifferent angles of view. A 3-dimensional model of the user's face isthen generated using the obtained images and the user's facial shape isdetermined using the generated 3-dimensional model. The determinedfacial shape is compared with user profile information stored in memory.The user profile information includes data relating to the facial shapeof authorized users. The comparison with the stored user profileinformation results in a determination as to whether the determinedfacial shape matches the profile information stored in the memory.Access to the system is granted when it is determined that the obtainedfacial shape information matches that of a stored authorized user'sprofile.

Other objects and features of the present invention will become apparentfrom the following detailed description considered in conjunction withthe accompanying drawings. It is to be understood, however, that thedrawings are designed solely for purposes of illustration and not as adefinition of the limits of the invention, for which reference should bemade to the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings wherein like reference numerals denoted similar elementsthroughout the several views:

FIG. 1 is a front view of a display monitor having recognition camerasaccording to an embodiment of the invention;

FIG. 2 is a front view of a display monitor having recognition camerasaccording to another embodiment of the invention;

FIG. 3a is a picture of a user taken by one of the cameras with aprojected light according to an embodiment of the invention;

FIG. 3b is a picture of the user taken by the other of the cameras withthe projected light according to an embodiment of the invention;

FIGS. 4a, 4 b and 4 c are three-dimensional representations of thesurface reconstruction of the user's face shown as a cloud of datapoints from various angles;

FIG. 5 is a normal picture of the user's face used to obtain facialtexture of the user; and

FIG. 6 is a flow diagram illustrating the method of recognizing anauthorized user according to the invention.

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS

FIG. 1 shows a first embodiment of the user recognition system accordingto the invention. In this embodiment, the display monitor 12 includes alight source 18 a and two charged couple device (CCD) cameras 14 a and16 a. Cameras 14 a and 16 a may be any other suitable known camera, suchas, for example, digital still or video cameras. The light source 18 aprojects a structured light onto the target, which in this case is theuser's face as they are situated in front of display 12. The use ofstructured light projected onto the user's face facilitates the facialreconstruction process and makes such process more reliable by providingan accurate frame of reference in the imaging system. A computer orcentral processing unit can be local to display monitor 12 or can bepart of a network of connected computers. Cameras 14 a and 16 a andlight source 18 a are connected to the computer 20 (either directly, orindirectly through a network of connected computers—e.g., a local areanetwork, or a global network such as the internet).

In another embodiment, the camera's and light projection apparatus maybe incorporated into a hand held device such as, for example, a mobiletelephone, a personal digital assistant (PDA). One of ordinary skillwill recognize that current 3 dimensional scanning circuitry is smallenough for incorporation into such devices and may be used as theimaging cameras for such hand held apparatus. The hand held device iscapable of communication with a global network of computers (e.g.,internet), either through wireless or wired protocol, such thatprocessing of the received image and 3-dimensional reconstruction can beperformed and re-transmitted to the hand held device.

FIG. 2 shows another embodiment in which cameras 14 b and 16 b and lightsource 18 b are integrated into the display monitor itself. Computer 20includes a memory that can be any suitable memory storage device capableof retaining user profile information that includes 3-dimensional imagesof the user's face and facial texture information relating to theauthorized users of the system. Examples of suitable memory deviceswould include random access memory (RAM), read only memory (ROM),erasible programmable read only memory devices (EPROMS), and any othermagnetic or optical media such as hard and floppy disk drives or CD-ROMS(read only, recordable, writable or otherwise).

FIGS. 1 and 2 show the respective cameras 14 a, 16 a and 14 b, 16 b onopposing sides of display 12. In accordance with an embodiment of theinvention, the cameras should be aligned in the same horizontal planewith respect to each other. The vertical position of the cameras withrespect to the display is not of integral importance and may be changedprovided both cameras remain on the same horizontal plane, and theangles at which the still images are obtained by the cameras areslightly different with respect to the user being imaged. As the twocameras are placed in different places along the same horizontal plane,the points closer to the camera (i.e., on the user's face) appear tomove left and right in the images, more than the points that are placedcloser to the background of the user. Other contemplated embodiments mayuse a single camera capable of movement between two predetermined pointsfor obtaining images at two predetermined angles of view. It isimportant that camera's 14 a, 16 a, and 14 b, 16 b are positioned sothat the user's face is within the focusing area of the respectivecameras when in a position to access the system. The cameras may also beprovided with horizontal, vertical and zooming capabilities in order toenable accurate obtaining of the images of the user's face. Theoperation of cameras 14 a, 14 b and 16 a, 16 b and light source 18 a, 18b will be described in conjunction with the flow diagram of FIG. 6.

Upon activation of the computer 20, or upon detection of an attempt toaccess computer 20 of privileged information contained thereon, lightsource 18 a, 18 b projects structured light onto the user's face (step30) and each camera 14 a, 14 b and 16 a, 16 b obtains an image of theuser's face from its respective angle point of view (step 32).Structured light is a grid of light that provides image reference pointsfor use in the surface reconstruction of the 3-dimensional model of theuser's face. According to the invention, the structured light may begenerated by any light source capable of being imaged (i.e., seen) by aCCD camera. Examples of such light are, LASER light, visible whitelight, and infrared light. When using LASER or infrared, considerationmust be made as to the effect, if any, the projected light may have onthe user's eyes. Low levels and durations of LASER light and infraredlight are not dangerous to a person's eyes, and as such provide otherlight source alternatives.

The use of structured light also prevents the possibility of trying touse mirrors and two-dimensional photographs in an attempt to fool thesystem into thinking the authorized user is present. The structuredlight prevents such attempts by bending with the 3-dimensional contoursof the user's face. When mirrors and 2-D photos are used, the structuredlight will not bend on the contours of the user's face, therebyindicating to the system that the user is not, in fact, sitting in frontof the computer. The two cameras take photos of the user from twodifferent predetermined angles with a slight delay between the first andsecond image obtained. This delay is dependent on the camera systemsused, and for example when using a video camera, may be decreased toonly a few milliseconds. FIGS. 3a and 3 b show the first and secondimages, respectively, obtained with the structured light projected ontothe user's face.

Once the images are obtained at step 32, the computer 20 or otherprocessing device generates a 3-dimensional model of the user's head(step 34). The process of generating a 3-dimensional model therefrom isreferred to as surface reconstruction. The 3-dimensional model generatedat step 34 is performed using any suitable known surface reconstructionalgorithm. Examples of such surface reconstruction techniques can befound in U.S. Pat. Nos. 5,202,928, 5,671,343, 5,694,483 and 5,819,016,each which are incorporated herein by reference. FIGS. 4a-4 c show theresult of the surface reconstruction as a cloud of data points. Theaccuracy obtained in the cloud of data points can be user prescribed tobe better or worse depending on the usable processing power (e.g. ofcomputer 20) and the level of accuracy desired.

Once the 3-dimensional model has been generated, a normal picture istaken of the user's face (FIG. 5) which provides a facial texture bitmap representation of the user. The 3-D model and the facial texture,information of the user are then compared with a previously created userprofile stored in a memory contained within computer 20 (step 40). Ifthe 3-dimensional data points match the points in the profile accuratelyenough and the facial texture bit map sufficiently matches the storedprofile, access is granted (step 42) to the system or specific files. Ifthe comparison does not result in sufficiently positive identificationof the user, access is denied (step 44).

Other features of the present invention may include a profile adjustmentprocedure performed during the match step 40. An example of when suchadjustment would be necessary would be when the user's facial shape haschanged for reasons such as, gaining weight or aging. In theseinstances, the user's profile may be updated during the match step toensure correct recognition of the user for future applications. Forexample, every time the user is granted access, the system will recordthe newly scanned model and add it to the previous stored profile suchthat the current stored profile is an averaged profile. Thus, the storedprofile will evolve with the changes in the user's appearance, andtherefore never deny access if the change in the user's appearance isnot too far from the average profile stored. In the event that theuser's appearance has changed drastically since their last accessattempt, for example, if the user has gained 100 pounds, they will berequired to contact a system administrator for purposes re-programmingthe system to recognize the user and store a new profile for the user.

While there have shown and described and pointed out fundamental novelfeatures of the invention as applied to preferred embodiments thereof,it will be understood that various omissions and substitutions andchanges in the form and details of the devices illustrated, and in theiroperation, may be made by those skilled in the art without departingfrom the spirit of the invention. For example, it is expressly intendedthat all combinations of those elements and/or method steps whichperform substantially the same function in substantially the same way toachieve the same results are within the scope of the invention. It isthe intention, therefore, to be limited only as indicated by the scopeof the claims appended hereto.

I claim:
 1. A method for recognizing a user comprising the steps of:obtaining at least two 2-dimensional still images of the user from atleast two different angles of view; generating a 3-dimensional model ofthe user's face from the obtained images; determining the user's facialshape using the generated 3-dimensional model; comparing the determinedfacial shape with profile information stored in memory, the profileinformation comprising data relating to the facial shape of authorizedusers; determining whether the determined facial shape matches theprofile information of any of the authorized users stored in the memory;granting access to a system when the determined facial shape matches theprofile information of one of the authorized users stored in the memory,thereby indicating recognition and authorization of the user; andupdating the profile information of the one of the authorized users withthe determined facial shape after each occurrence of said step ofgranting access such that the updated profile information is an averageof the profile information previously stored in memory and thedetermined facial shape.
 2. The method set forth in claim 1, whereinsaid step of obtaining at least two 2-dimensional still images furthercomprises: providing two cameras directed at the user's face; providinga light source for projecting structured light onto the user's face; andcapturing the at least two still images using the cameras.
 3. Anapparatus for recognizing a user's identity during an attempt to accessa system, comprising: at least one charged coupled device (CCD) cameraadapted to obtain at least two still images of the user's face from atleast two different predetermined angles of view and to obtain at leastone still image of the user's face; a light source for projectingstructured light on the user's face for use in obtaining said at leasttwo still images of the user'face; memory means for storing user profileinformation relating to authorized users of a system; and processingmeans connected to said at least one CCD camera and said light sourcefor generating a 3-dimensional model of the user's face using the atleast two still images, and for generating a facial texture bit map ofthe user's face using the at least one still image, said processingmeans comprising means for comparing the 3-dimensional model and thefacial texture bit map to the stored user profile information containedin said memory means, means for enabling access to the system when thegenerated 3-dimensional model and facial texture bit map match a userprofile stored in said memory means and means for updating the userprofile information with the 3-dimensional model, wherein the updateduser profile information is an a average of the previous user profileinformation and the 3-dimensional model.
 4. The method set forth inclaim 1, wherein said step of determining whether the determined facialshape matches the profile information further comprises denying accessto the computer system when the user's determined facial shape does notmatch any of the profile information stored in memory.
 5. The method setforth in claim 2, further comprising the steps of: obtaining at leastone additional 2-dimensional still image of the user's face; andgenerating a facial texture bit map of the user's face using theobtained 2-dimensional still image, wherein said step of comparingfurther comprises comparing the generated facial texture bit map withthe user profile information stored in the memory.
 6. The method setforth in claim 1, wherein said step of generating a 3-dimensional modelis performed using a surface reconstruction algorithm.
 7. The method setforth in claim 1, wherein said step of obtaining is performed by acharged couple device (CCD) camera capable of movement between twopredetermined positions.
 8. The method set forth in claim 1, whereinsaid step of obtaining is performed by at least two charged coupledevice (CCD) cameras.
 9. The method set forth in claim 1, wherein saidstep of obtaining is performed by a digital camera capable of movementbetween two predetermined positions.
 10. The method set forth in claim1, wherein said step of obtaining is performed by at least two digitalcameras.
 11. The method set forth in claim 2, wherein said step ofobtaining is performed when the user is positioned within apredetermined focusing area of said two cameras.
 12. An apparatus forrecognizing a user's identity comprising: a light source for projectinglight at the user's face; a first camera directed toward the user's faceand adapted to record at least one still image of the user from a firstpredetermined angle of view; a second camera directed toward the user'sface and adapted to record at least one still image of the user from asecond predetermined angle of view; memory means for storing userprofile information relating to authorized users of a system; andprocessing means connected to said light source and said first andsecond cameras for processing the still images obtained by said camerasand generating a 3-dimensional model of the user's face and forcomparing the generated model with the stored user profile informationto determine whether the user is authorized to access a system, saidprocessing means comprising means for granting access to the system whenthe generated model matches the profile information of one of theauthorized users stored in the memory means, thereby indicatingrecognition and authorization of the user, and means for updating theprofile information of the one of the authorized users with thegenerated model after each grant of access by said means for grantingaccess such that the updated profile information comprises an average ofthe generated model and the previously stored profile information. 13.The apparatus in accordance with claim 12, wherein said light sourceprojects structured light onto the user's face to facilitate thegeneration of the 3-dimensional model.
 14. The apparatus in accordancewith claim 12, wherein said first and second cameras comprise chargedcouple device (CCD) cameras.
 15. The apparatus in accordance with claim12, wherein said first and second cameras comprise digital cameras. 16.The apparatus in accordance with claim 12, wherein said memory meanscomprises at least one selected from a group consisting of RAM, ROM,EPROM and a magnetic storage media.
 17. The apparatus in accordance withclaim 12, wherein said processing means comprises a computer, saidmemory means being contained within said computer.
 18. The apparatus inaccordance with claim 12, wherein one of said first and second camerasis adapted to obtain a 2-dimensional still image of the user's face,wherein the obtained 2-dimensional still image of the user's face isused to determine the user's facial texture, the determined facialtexture being used in conjunction with the generated 3-dimensional modelto determine whether the user is authorized to access the system. 19.The apparatus in accordance with claim 13, wherein said light sourcecomprises at least one selected from a group consisting of white light,Laser light and infrared light.